Click any tag below to further narrow down your results
Links
David Breunig's talk on how context engineering emerged after the long-context push of 2024, why bigger context windows introduced new failure modes, and the six patterns builders now use to manage what goes into a model's context. It's a useful map of where the practice has been and where it's headed.
- Million-token context windows didn't solve the problem. Long contexts add room for errors as well as information, leading to failure modes like context poisoning, distraction, confusion, and clash.
- Gemini 2.5 playing Pokémon shows poisoning in practice: a fact baked into its weights (the tea bribe from the older game) got stuck in its context and cost it hours. Performance also degraded past roughly 100,000 tokens, with the agent repeating old actions instead of planning new ones.
- Breunig's research cites shows instruction-following drops as bad examples pile up in context, affecting every model tested, including smaller ones that fall below half adherence after a few contradictions. The six-tool toolbox (augmentation, loadout, quarantine, pruning, offloading, summarization) has stayed stable as these problems emerged.
@sairahul1 on X
A post from @sairahul1 recommends a lecture by former Google Chief Scientist Jeff Dean that walks through the full AI engineering stack, from building LLMs from scratch to one person coordinating 100 agents. It also points to a linked course on production agent systems covering orchestration, RAG, evals, context engineering, and inference.
- The lecture covers the whole stack, from training LLMs from scratch up to coordinating large numbers of agents.
- The author says it suits both newcomers who haven't used a model and engineers already shipping agent systems.
- The linked course argues developers should learn how production agents work rather than take another prompt engineering course.